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ABSTRACT 

The concept of an educational goal indicator (EGI) 
was devised as a measure of a post-educational condition or behavior 
that an educational system is supposed to produce or affect. In this 
sense it is intended to be a "true" measure of the schools • 
accomplishments. A search was made, as part of the New Jersey Needs 
Assessment program to identify potentially useful EGI»s and to 
determine what steps might be taken to develop others. The search was 
limited to those indicators for which data were already available in 
existing sources — primarily various publications and a public opinion 
survey being conducted, into which were introduced several items that 
could be the basis for EGI»s. The results of the search are discussed 
xmder the headings of the 11 outcome goals defined by part of the 
Needs Assessment program. The findings suggest a distinction between 
two types of indicators, intentional and incidental. Intentional 
indicators are those based on data gathered expressly for the purpose 
of measuring the degree of attainment of educational goals; 
incidental indicators are based on data gathered for some other 
purpose. A comparison of the two types based on ten characteristics 
of good EGl*s reveals that intentional indicators are preferable. 
(KM) 
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A common way of finding out how well schools are accomplishing the 
goals of education is by administering tests to students* Yet tests have 
certain deficiencies for this purpose* Test scores may tell us whether 
knowledge, skills, and understandings have been acquired by students, but 
they do not tell us whether the knowledge, skills, and understandings 
are actually put to use by people when they are no longer students « That 
is, they reveal nothing about the extent to which the schools are bringing 
about the life-long attitudes and behaviors that are the ultimate aims of 
education* Secondly, there are some kinds of educational goals that are 
not satisfactorily measured by tests or that may not be observable in 
schools at all—for example, adequate fulfillment of parental responsibilities, 
or conscientious participation in political decision-making. Thirdly, the 
widespread administration of tests sometimes gives rise to the fear that 
the schools are devoting inordinate effort to training students for taking 
the tests, thus narrowing and distorting the educational process • Finally, 
test scores have an air of "artificiality" about them; they appear to be 
numbers that have no relation to "real-life" activities and so are unconvincing 
to many people as a measure of meaningful educational progress* 

The concept of an "educational goal indicator" (EGI) was devised to 
overcome these deficiencies* An EGI is a measure not of what students are 
acquiring in school but rather of a post- educational condition or behavior 
that an educational system is supposed to produce or affect* In this 
sense it is intended to be a "true" measure of the schools' accomplishments* 



Indicators need not be limited to what can be observed in schools; they 
can cover the entire range of a<?ult life. They avoid the dangers of 
"teaching to the tests"; since an indicator would be based on behavior 
that is significant in its own right, "teaching to the ir^dicator" would 
be exactly what the schools would be expected to do. And because 
indicators would be expressed in terms of actual everyday behaviors and 
events, they would be readily understood and directly meaningful. As an 
illustration which sums up these features, the proportion of the eligible 
population in New Jersey who vote in a presidential election would be an 
indicator of the degree of attainment of the goal that "the public schools 
of New Jersey should help every person in the state to become an effective 
and responsible contributor to the decision-making processes of the 
political and other institutions of the community, state, country, and world." 

The example also suggests, however, that there are important qualifica- 
tions on the use of EGl's. First, no single indicator can be regarded as 
an adequate measure of goal attainment (2tzioni and Lehman, 1967). Educational 
goals are of necessity stated in broad terms; they refer to many different 
kinds of behavior. Effective and responsible contributions to decision- 
making processes can iuclude not only voting — and voting not only in 
presidential elections- -but also keeping well informed, engaging in discussion 
of political issues and candidates, attending meetings of governing bodies, 
writing letters to legislative representatives, running for office, and 
numerous other activities, not to mention forms of participation in the 
decision-making processes of institutions other than the political ones, such 
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as voluntary organizations. On the other hand, EGl's cannot be taken 
as unequivocal measures of the effectiveness of the educational system, 
and particularly not of the educational system of a single state. Many 
of New Jersey's residents -eceived all or part of their education outside 
the state, and many people who were educated in New Jersey now reside 
elsewhere. (Moreover, the data on which EGI*s are based do not always 
identify even the state of residence.) In addition, EGI's are also 
affected by many factors more or less beyond the control of the schools- 
factors such as conditions in the home and community, economic trends, 
events on the national and international scene, etc*^ Specific illustrations 
of these and other "extraneous" factors will be given later. 

Nevertheless, in view of their apparent advantages, an effort was made, 
as part of the New Jersey Needs Assessment program, to identify potentially 
useful EGl's and to determine what steps might be taken to develop others. 
A "potentially useful EGl" was defined for this purpose as a repeatable 
measure of a condition or behavior that occurs outside of school but that it 
is reasonable to expect can be affected by experiences within the formal 
educational system of preschool, elementary, secondary, vocational, and 
adult-education (but not college or university) programs. The criterion 



It should be pointed out that this is also true of test scores. 
More and more, it is coming to be recognized that what students learn 
in school is much influenced by what is happening outside the schools. 
The difference between EGl's and test scores is, in this respect, a 
matter of degree. 



of repeatability was Included to insure that the measure could be taken 
at two or more occasions to determine the degree of progress toward 
goal attainment chat was being made* 

In order to keep the search for indicators within a realistic frame- 
work, it was limited to those for which data were already available in 
existing sources* (One consequence of this decision is that the £GI*s 
refer to different dates*} Since the interest was, of course, in EGI*s 
for New Jersey, the sources were chiefly publications of various state 
agencies, although publications of some federal and private agencies were 
examined as well* In addition, a public-opinion survey was being conducted 
by Opinion Research Corporatlor (ORG) in connection with the Needs Assess- 
ment program, and the opportunity was used to Introduce into the survey 
Instrument a number of items which could be the basis for £GI*s* This 
report is an account of the results of the search, organized under thu 

headings of the outcome goals which emerged from the "Our Schools" project 

2 

of the Needs Assessment program* 



By their nature, £GI*s (and test scores, too, for that matter) are 
relevant only to outcome goals, not to process goals* Attainment of the 
latter type of goals must be measured in terms of the activities of the 
schools, since process goals by definition do not refer explicitly to any 
effects upon students* 
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Outcome Goal I 

The public schools of New Jersey should help every person 

in the state to acquire basic skills in obtaining information, 

solving problems, thinking critically, and communicating 

effectively. 

Mastery of the traditional three R's can be regarded as a necessary 
part of this goal, but clearly it also involves the skills of listening 
and speaking, and — particularly with reference to "solving problems" and 
"thinking critically" — the applicatioigi of reasoning and sound judgment. 
However, the data located as potential EGI*s for this goal tend to 
emphasize verbal and mathematical skills and are thus very limited 
measures of its attainment* Even at that, they have severe defects* 

One figure frequently cited as an indicator of mastery of the basic 
skill§ is the proportion of men who qualify for military conscription 
by their scores on the pre-induction mental test. This test, known as 
the Armed Forces Qualifying Test (AFQT) , consists of 100 multiple-choice 
items equally distributed among the content areas of vocabulary and verbal 
concepts, arithmetic reasoning and computation, spatial relations, and 
mechanical ability, and the test scores have been shown to be related to 
Itivel of education (Karpinos, 1967). Qualified examinees are those whose 
scores are in approximately the highest 69 percent of all scores, plus 
some whose scores are lower than that. (For complete details, see Office 
of the Surgeon General, 1967, pp. 8-11.) 

Table 1 shows the percentages of examinees who qualified on this 
test in New Jersey and in the United State, as a whole, for 1965 and 1968. 
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It Indicates that, while the qualification rate for New Jersey youths 
Increased somewhat over the three-year period, the Increase was less than 
that for th3 entire country, so that by 1968, New Jersey youths qualified 
slightly less often than did American youths as a whole* Black examinees 
qualified at a far lower rate than white examinees In both years. In 
the state and In the nation, and lost even more ground In New Jersey, 
relative to members of the same racial group, than did white examinees 
between 1965 and 1968. 

The objection may be made that the AFQT Is still a test and 
therefore does not meet the criteria of an Indicator. However, It Is a 
test given outside of school, and It does have Important "real-life" 
consequences. But there are a number of other, more serious objections. 
The most obvious of these Is that the measure Is applied only to luales. (It 
Is applied only to young males, too, but that could be regarded as an 
advantage, since It Is thereby more directly reflective of the system of 
elementary and secondary education.) Secondly, the number and kinds of men 
who take the test, and the qualifying level that Is set each year, depend 
upon legislation, the rules governing exemptions, trends In voluntary 
enlistments, the needs of the armed forces relative to the number of men In 
the eligible age group, and even attitudes toward service In the armed forces- 
all of which change from year to year In ways that are largely or wholly 

q 

extraneous to the educational system. Thus, the changes In the Indicator 



For a history of some of these changes between 1963 and 1966, see 
Office of the Surgeon General, 1967, pp. 10-11. 
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may not be a result of anything the schools are doing. Finally, If 

conscription Is replaced by an all-volunteer armed force, as seems 

likely In the ear future, the measure will not be repeatable. Though 

these objections appear to be specific to this particular Indicator, 

they Illustrate a general problem with EGI*s: Their magnitude Is subject 

to variations stemming from changing social policies and social conditions » 

variations which may have far greater effects than the work of the schools. 

Another measure that has been suggested as an indicator of mastery 

of basic skills is the proportion of people who pass the written part of 

the test for a driver's license among all those who take the written test. 

This is again a test-based indicator, though it, too, has more than purely 

academic consequences , and it is taken by males and females alike over a 

wide range of ages. In the 1971-72 fiscal year, the proportion in New 

4 

Jersey was 77.1 percent. 

This would be a valid indicator of basic skills only on the assumption 
that the ability to pass the test depends primarily on the ability to read 
and understand the written questions. Actually, of. course, it depends at 
least as much on one's knowledge of the rules of the road* Such knowledge 
might be regarded as a "basic skill" In a society as heavily reliant upon 
the automobile as ours is, but the ordlng of the goal statement makes it 
seem unlikely that this was intended. Furthermore, it has been pointed out 



Data kindly provided by the Division of Motor Vehicles. Neither 
comparable figures for previous years, nor figures for the United States 
as a whole, are readily available* In the 1968 calendar year, 41 percent 
of license applicants were more than 25 years old* 
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thaty precisely because of the pc;rvasive Importance of having a driver's 
license^ states are probably rather lenient in issuing them, so that 
passing the test may not be a truly valid measure of any skill at all 
(Hahn, 1970, p. 96). At the very least, it must be said that the 
proportion passing is influenced by varying enforcement policies. On 
the other hand, there may well be many people so doubtful of their 
ability to pass the written test — or perhaps the vision or driving 
parts — that they do not even tdke it, so that their scores never enter 
into the figures. It should also be noted that 25 percent of those 
who passed New Jersey's written test in 1971-72 took it in a language 
other than English* 

Still another test-based measure that might be thought of as an 
EGI is the mean score achie/ed by students taking the Scholastic 
Aptitude Test (SAT) of the College Entrance Examination Board. Table 2 
gives the data for 1968-69, but once more, the figures are much less meaningful 
than they might appear to be. Like the driver !s test, the SAT is taken 
by a self-selected group. Even to label the scores as being those for 
the "United States" and for "New Jersey" is misleading, since the test-takers 
are by no means — though they are sometimes thought to be— the total twelfth- 
grade student body; the 59,000 New Jersey tudents who appear in Table 2, 
for example, represent only 60 percent of -he 99,000 young people who 
graduated from the state's public and non-public secondary schools in June, 
1969. Nor are they a random sample, for the test is, of course, taken mostly 
if not excusively by students planning to attend college. Thus, mean SAT 
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TABLE 2. MEAN SCORES OF HIGH-SCHOC C 
ON THE SCHOLASTIC APTITUDE TEST, l;>vo-69 



Boys 



Girls 



U.S. 



N.J. 



U.S. 



N.J. 



Number taking the 
test 

Mean SAT-Verbal score 

Mean SAT-Mathematical 
score 



496,823 32,281 
458 457 



510 



502 



414,845 
464 

466 



26,763 
469 

464 



Sources: For the U.S., College Board S core Reports; A guide for counsellors 
and admissions officers (College Entrance Examination Board, 1969), 
p. 24. Data for N.J. were provided by, and are printed with the 
permission of, the College Entrance Examination Board. 
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score levels are heavily Influenced by rates of persistence In high 

o distributions of aspirations for college, the kinds of colleges 
applied to, and the number and kinds of colleges requiring the SAT 
for admission — and these factors vary. In complex and largely unknown 
ways, from state to state and from year to year. (The changes may be 
especially great In New Jersey over the next few years, with the rapid 
Increase of enrollments In the state's higher educational system.) 
Certainly It Is unlikely, then, that differences between state and nation 
as small as those shown In Table 2 will be much affected by Improvements In 
the quality of formal Instruction. Moreover, the SAT Is a much more "purely 
academic" Instrument than either of the other two tests discussed above; 
and the very use of the SAT score as an Indicator might convey the Impression 
that entrance Into college Is a desirable educational goal In and of Itself. 
On all of these grounds, SAT scores are a particularly poor candidate for 
an EGI. (For further discussion of the use of test scores for Indicator 
purposes, see Schrader, 1968.) 

Another possible basic-skills Indicator that was considered was the 
number of errors of calculation made on Income-tax returns per 100,000 
returns filed by New Jersey residents. The usefulness of such a measure 
would be diminished by changes In the tax-return forms (which might affect 
the compreaenslblllty of the Instructions), and by the facts that many 
residents do not file returns (their number and characteristics varying with 
the tax laT/s) and that some of those who do file returns do not prepare 
them themselves. In any case, the requisite data for this Indicator are 
not presently available; Indeed, the Internal Revenue Service does not 
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even compile such Information, though this difficulty might be over- 
come If Ne'/^r Jersey should enact Its own Income-tax law. 

Perhaps the most promising Indicators In the basic-skills area 
come from the ORG survey. As part of the survey, respondents were given 
a list of 65 activities and asked whether they engaged In them "freq- 
uently or regularly," "occasionally or every now and then," "rarely or 
only once or twice," or "never," These activities were specifically 
designed as potential EGl's. The ones intended primarily as basic-skills 
indicators are listed in Table 3, together with the percentages saying 
they engaged in them "frequently or regularly" among respondents who 
were 16-29 years old (and so had completed their normal education 
relatively recently) and those who were 30 years old or more. It is 
interesting to note that, despite substantial differences between these 
two categories on some of the items, the averages for all the items together 
were almost the same. The capability for averaging items, since the 
response format for all is the same,^ and the capability for making 
distinctions along such dimensions a: age, sex, racial/ethnic group, length 
of attendance in the state's schools, etc., are among the advantages of 
this type of indicator.^ 



For other items in the ORC survey that were not primarily Intended as 
basic-skills indicators but could be included among them, see Table 5, items 
5 and 6; Table 6, itemt; 6,8, and 9; Table 8, items 10 and 11; Table 11, item 
2; and Table 14, all items. 

^ORC's own report of the survey classifies respondents on the indicator 
items into those who had attended New Jersey public schools for six years or 
less and those who had attended them for more than six years ("Goals for 
Elementary and Secondary Schools in New Jersey," pp. 93-108). On the 
whole, the differences were rather small* 
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The Items In Table 3 may appear to be Inadequate for EGI's because 
they are measures of frequency rather than of effectiveness. They tell 
us, for exampl^^, how often people look up a number In the telephone 
book, but not how long it takes them, how easy they find it to be, or 
whether they locate the correct number. However, it seems not an unreason- 
able assumption that frequency and effectiveness are related: the longer 
it takes a person to find a number in the telephone book, the harder it 
is for him, and the less often he locates the right number, the less 
frequently he is apt to make the effort at all, turning instead to another 
person, or to directory assistance, to find the number for him» 



Outcome Goal II 

The public schools of New Jersey should help every person 
in the state to acquire a stock of basic information 
concerning the principles of the physical, biological, and 
social sciences, the historical record of human achievements 
and failures, and current social issues. 

It is difficult to imagine any EGI*s for this goal other than those 
that could be obtained from questions directed to a sample of the popu- 
lation specifically for indicator purposes, to learn just what information 
they have. Even at that, a single survey could not cover more than a smal. 
part of the range of knowledge implied in the goal statement. However, 
groups of experts in each field might be able to agree on a set of items 
that would "represent" the vital information in that field; then, by 
confining the survey in any one year to a few fields, and perliaps also by 
using item-sampling techniques, it might be feasible to measure the extent 
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TABLE 3. POTENTIAL INDICATORS FOR OUTCOME GOAL I FROM THE ORC SURVEY 



Among respondents whose age was: 
16-29 years 30 years or more 



Percent who "frequently or regularly": 

1. Discuss their point of view 
with others who hold different 

opinions 63 46 

2. Balance their checkbook 30 50 

3. Look up someone's number in 

the telephone book 43 41 

4. Find out how to get somewhere 

by looking at a map 35 41 

5. Use a dictionary 29 37 

6. Use an encyclopedia 30 26 

Mean 38 40 



Source: Computer printouts provided by Opinion Research Corporation, 
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of possession of a "stock of basic information" over a four- or five- 
year period. This is essentially the procedure used by the National 
Assessment of Educational Progress (Womer, 1970), and its work would 
be a valuable model* 

The ORG survey asked a number of questions dealing with respondents* 
activities aimed at acquiring information, which will be discussed below 
under Outcome Goal XI. It may be pointed out here that they do not 
reveal what was actually learned in these activities, but inducing 
people to seek information is one way of helping them to acquire it, so 
the frequency of engaging in such activities can properly be regarded as 
an indicator for Outcome Goal II. A few ORG questions did tap specific 
items of information about politics and health, and the results will be 
given under Outcome Goals III and VII, respectively. 

Outcome Goal III 

The public schoolt. of New Jersey should help every person 
in the state to become an effective and responsible contributor 
to the decision-nnaking processes of the political and other 
institutions of the community, state, country, and world. 

The most obvious indicator for this goal is voting behavior, but it 
is by no meanis as simple a measure as it might seem. For one thing, 
registration is a necessary prerequisite to voting; not only does this 
raise the question of whether the base for determining the proportion 
voting should be the total population of voting age or only the number of 
registered voters, but it also suggests that the proportion of the voting- 
age population which is registered to vote might be considered as an 
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indicator in its own right. Furthermore, eligibility to register and 

vote depends upon such "extraneous" matters as age and residence 

requirements; the former was changed only recently and the latter are 

changing now and also vary among the states (thereby affecting comparisons 

between Nej Jersey and the rest of the country). Finally, there is the 

problem of selecting indicators from among the votes for a host of 

elective offices and referendum questions. Table A shows that, considering 

only the votes in general elections for president, governor, and senator, 

and on two referenda, the magnitude of potential voting-behavior 

indicators for New Jersey ranged from 42 percent to 87 percent in three 
7 

recent years. 

The proportion of registered voters who vote for president, and 
perhaps even for a state-wide office such as governor or senator, is 
probably already too high to serve as a useful indicator of educational 
progress. It is more likely that the percentage of the voting-age 
population that voted can be increased, and this also has the virtue of 
reflecting both the proportion who register and the proportion of 
registrants who vote — although, by the same token, it also incorporates 
the uncertainties associated with eligibility requirements. In addition 
to votes for president and state-wide offices, some lower-level offices 
might be used as well; Congressional representatives would seem to be most 
suitable, because they are elected in all districts at the same time. 



Voting in primary elections is probably not an acceptable indicator, 
because it demands the declaration of a party affiliation, which might 
not be universally regarded as a civic obligation. 



ERiC 



16 



CM 



00 
NO 



as 
so 

ON 



00 
vO 









ft 

u 




























•H Q) 




























CO CO 














4J CO 




























0 


vO 


























CO c 














s s 














CO ,0 














M 














H 














M 














0 














4J 


CV| 


0 


rH 








!3 


00 


vO 










c 




o> 


1— 1 


* 


CO 








rH 








CO 




























•U 






CO 














Q) 














4J 








'd 






CO 








tH 




CM 


4J 








CO 


00 


vO 


CO 








Q) 














M 






'd 








p4 




















































































(1) 






* 


vO 
















00 


•H 


0 








w « 


vO 


CO 


00 








1 




Q) 










rH 




M 






0 








p4 


































CO 








0 0) 






M 














U 








^ CO 






CO 








CO CO 














> 


00 




< 








u 














Q) 'd 






rH 








CO c 






CO 








S S 








* 






0 rO 






•H 


• 




CO 








4J 

CO 






r% 








•H 














iJ 








u 






CO 








0 






iJ 














CO 








M 


CO 


CO 










0) 




*n 










> 






(0 








0 




















CO 
























































<U 


* 






0) 






•C 








•d 






4J 


* 






•H 




so 










CO 


00 


so 


*W 








0) 






0 








u 


































CO 














0) 












































• 






0) 








CO 


• 




•d 






• 


• 


00 


OS 


tH 


o\ 


csi 


CO 




rH 




CO 


00 


so 






iH 




0) 






p 








M 




















** 












* 


CO 




























iJ 

CO 






0) 




•d 


Q) •d 


4J 






60 0) 




0) 


60 Q) 


CO 






CO M 




u 


CO ^ 






o 


1 0) 




0) 


i 0 


•d 






60 






60 > 


Q) 






d CO 




CO 'd 


c 


4J 






•H tH 




tH 0) 




•H 




tH C 


<U 60 




60 iJ 


4-1 CO 








0 Q) 




Q) 0 


0 43 






CO ^ 


> M 




u > 


> ^ 
















U 










0 










0 0 






0 0 


MH 




O CO CO 


•H 




"•I 








O4 C 






u 




CO 




to 0 


d CO 




C CO 


d CO 


0) 




-« 5 


0) H 




0) u 


0) rH 


0 




CO H 


0 d 




0 0) 


0 d 


M 




U U r- 


M a. 




M U 








0 0 X 


0) 0 




0) 0 


0) 0 


0 




H > 0 


{l4 0. 




(l4 > 


p4 a. 


CO 



00 
00 



so • 

CO O4 

o> •» 

CO r% 
o> 

• rH 

04 • 



m 

M 

0) 

a 
> 

o 



0) 

o 

•H 
MH 
MH 
O • 
0) 

60 rH 
•H fa 

u 

d d 

•H O 
U 

Ph CO 
4J 

*J O 

d CO 

0) fa 



0) CO 

> 

o • 



d fN. 
o o\ 

U rH 
60 

d • 

•H CN 
(0 • 

eg > 
^ o 

^2; 



d 0) 

O CO 

•H M 

iJ Q) 

CO 
rH 

o 2; 

MH 

o> O 

SO 

rH iJ 

: CO 

A CO 

d MH 
Q) O 

Q) 

rH M 
Ck. CO 

d 0) 

CO M 
O 

rH 0) 
CO CO 

O <U 

O4 u 
CO 

MH 

• o 

H 

a 

• -H 

rH MH 
O MH 
> O 



CO O 
D 

iJ 

• A d 

O Q) 

rH M 

CO d 

CN 



H O 

0«*C CO ^ 

4J 0) <U 

M 0 

/-N MH d iJ 

CN O PQ M 

^ r% CO 

CVO> to Q* 



0*CN 4J 
04r^ d - 

0> •H Q) 



d 



d jo <u 
•» ij 'd 

0) Q) 0) 

O Q) 'd 0) 

•H -H CO 

MH > U 

MH 0) O 0) 

O ^ M 

iJ Ck. ^ 

60 > 

d MH CO 0) 

O iJ » 
CO 

CO -^CJ • 



P4 



ij •d 

d CO i 
•H a 



<r o 
> 



0 
O 

d 
o 
o 

M 
CO 

u 

0) 



o> CO d o 

M 0) r CO rH 

Q) 0 00 0) 

d • > 

M r—i • 0) 

0) Ck«Q 

> 0) 

_ o CO t a 

0 ^ 0) CN 0 

0) •• o 

> MH d d 

O O O > rH O 

^z; iJ 0) a 

CO 6OS2; M 

'd a d 

rH -H •H MH MH 

0) 4J ^ O M O 

rC CO (0 0) 

•H CO 0) ^ d 

d M ij 0 o 

O 0) CO 0) •H 

•H iJ iJ a CO 

u a o CO 0) iH 

a CO CO o > 

0) M CN MH 

rH O • O 

W ^ • CO 

O O M • 

rH r ^ M 0) CO 

(0 CO CO a 

M iJ CO iH 

0) CO O 0) CO 

d d CN M > CO 

0) CO I a d iH 

o d Ph 0) CO 

0) CO U CO 

0) U (0 iJ 

^ 0) 0) 0) CO 

iJ 0) iH 4S 4J 

jc3 M iJ r: 'd 

MH iJ 0) iJ d 

O CO MH CO CO 

MH O 

o • 



CO 

_ CO 

rH d iJ 

d CO M 

(0 0) o 

j5J 



MH ^ 

a 

M 

CO 
0) 
. (0 
O O 0) 



0) o 

o 

iH •d 

_ MH M 

u oJmh CO 

3 0)- 



0 

Q4 
o 

rH 
0) 

s; 



0 
o 
d 
o 
o 

M 

"H 

CO 

d 
o 



CO 

t 

Q) 
(0 

d 
o 

CJ 
MH 

o 



>% 

0) 

to 
o 

MH 

CO 
0) 

o 

M 

o 

CO 



ERIC 



17 



The use of referenduia questions Is more dubious, since these do not 
appear on the ballot with any regularity and, when more than one 
Is presented, It might be difficult to agree on which one Is to be the 
Indicator. However, an arbitrary choice could be made, such as the 
question that appears In the uppermost or lowermost posltlca on the 
ballot. The objection might be raised that some questions draw more public 
attention than others, but It Is precisely that kind of difference that 
the goal suggests shoulc! be overcome. The same can be said of votes for 
Congressional representatives In presidential and non-presldentlal years. 

It has been pointed out previously that an "effective and 
responsible" contribution to political declslon-*maklng processes Involves 
a great deal more than voting. Most people would probably feel, for 
example, that the good citizen Is well Informed about political matters. 
The ORG survey asked respondents If they could name the governor of New 
Jersey and Its two senators — a minimal sort of political knowledge. 
The percentages who were able to give the correct names are shown In part 
A of Table 5. While the great majority knew who the governor was, less 
than a majority could name either of the two senators, and of this group, 
only about half could name both. Older people were a little better Informed 
than younger people were by this measure. 

Part B of Table 5 shows the percentages of respondents In the survey 
who said they "frequently or regularly" engaged In a number of activities 
that are related to Outcome Goal III. Although some of the activities, 
such as giving money to a church or charity and doing volunteer work for 
a civic or social organization, might not be thought of as contributions 
to decision-making processes In a strict Interpretation of that term, 
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TABLE 5. POTENTIAL INDICATORS FOR OUTCOME GOAL III FROM THE ORC SURVEY 



Among respondents whose age was 
16-29 years 30 years or more 



A, Percent who could name : 

1. The governor of New Jersey 

2. One New Jersey senator 

3. Both New Jersey senators 



79 
17 
17 



86 
21 
27 



B. Percent who "frequently or regularly" ; 

4. Contribute money or time to help 
someone get elected to public office 

5. Send a message of support or protest 
to an elected public official 

6. Write a letter to an editor of a 
newspaper or magazine expressing 
an opinion 

7. Attend a local school board meeting 

8. Serve as an officer of a political 
or social organization 

9. Contribute to a church or charity 

10. Do volunteer work for a civic or 
social organization 

11. Give blood 

12. Pick up litter on the street 

13. Return glass, metal containers, or 
newspapers so that they could be 
re-cycled or used again 

Mean of items 4-13 



2 
8 

9 
35 

12 
2 
20 

15 
11 



3 
6 

12 
68 

17 
5 
31 

21 
18 



Source: Conqputer printouts provided by Opinion Research Corporation 
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they have been Included here because they seem to represent the 
"responsibility" of the citizen to do those things that allow the insti- 
tutions of his community and society to function. (The omission of any 
indicators of "world citizenship" is notable, and one would be hard put 
to invent them.) The last three activities in the table — giving blood, 
picking up litter, and re-cycling materials — ^may be regarded as 
extensions of this concept of responsibility, but perhaps they are too 
far removed from the intent of the goal to be suitable indicators* 

It is possible to suggest measures for some of these activities 
that would not rely on a survey; for example, elected officials could 
be asked to report the number of letters they receive expressing opinions 
on political Issues. This might be considered a more accurate measure 
than responses to a survey question — although the survey figures do not 
Indicate any great exaggeration of frequency; on the other hand, some 
officials might object to tabulating or reporting this kind of information. 

Outcome Coal IV 

The public schools of New Jersey should help every person 
in the state to acquire the knowledge, skills, and under*- 
standings that permit him/her to play a satisfying and 
responsible role as both producer and consumer. 

It is surprising that no indicators could be found in published 

sources which might be suitable for measuring the extent of attainment 

of the "producer" part of this goal, even though statistics on employment 

are so abundant. The usual employment figures, however, are actually not 

very helpful. The proportion of the labor force which is unemployed, for 

20 



example, depends overwhelmingly on economic rather than on educational 

Q 

conditions. It could be argued that the Intent of this goal Is 
that education should equip every person either with employable skills 
or with the ability to acquire employable skills, no matter what the 
economic conditions; but this Implies that the proper Indicator would 
be the proportion of the labor force which Is unemployed because of a 
lack of appropriate skills, and such a classification would be difficult 
to establish. The proportions of the labor force employed In different 
occupations are similarly affected primarily by economic (and technological) 
conditions; moreover. Outcome Goal IV Is presumably not meant to suggest 
that particular proportions should be engaged In particular occupations. 

More pertinent measures might be the percentage of applicants who 
attain a qualifying level on clvll-servlce examinations or who are 
successful In applying for occupational licenses where these are required. 
Data of this kind could not be located, and It must be pointed out that 
they would, once again, be test-based Indicators derived from the 
performance of self-selected groups. 

None of these data would. In any event, reveal whether Individuals 
were either satisfied or responsible In their roles as producers, and It 
Is those attributes which are the main emphasis of the goal. For such 
Information, as well as for Indicators of the same attributes among 
consumers, survey questions appear to offer the greatest promise. The 



o 

There are technical difficulties with the figure as well. To 
mention but one, some people have withdrawn from the labor force only 
because they have decided tha1^ they cannot find a suitable job. 
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ORC survey contained several possibilities, which are ulsplayed In 

Table 6. Most of the producer-role lt«ns probably reflect a combination 

of satisfaction and responsibility; It may be assumed, for example, 

that the person who puts In extra hours at work feels a sense of 

obligation to his job yet would be less likely to do It If he didn't enjoy 

his work as well (though It must also be recognized that the opportunity 

for putting In extra hours depends to some extent on economic conditions). 

Recommending one's place of work to someone else, on the other hand. Is 

chiefly an Index of satisfaction. If It were considered Important to 

distinguish more sharply between these two aspects of work, future 

surveys could Include a straightforward question about job satisfaction, 

while employers or supervisors could be asked about the degree of 

9 

responsibility shown by persons In their charge. 

The consumer-role Items In Table 6 are chiefly measures of responsi- 
bility, although It may well be that the person who regularly engages 
In the activities shown Is also a more satisfied consumer. (Item 13 In 



In a separate part of the ORC survey, 114 "supervisors or 
evaluators of employees" were asked for comments about their experiences 
with the graduates of New Jersey public schools. This was not a repre- 
sentative sample, and the results cannot be precisely quantified; 
nevertheless. It Is worth mentioning that the survey report noted there were 
"many complaints about the fact that people are not sufficiently motivated 
on the job, that they are too demanding and are not interested In 
learning or putting In a full day's work" ("Goals for Elementary and 
Secondary Public Schools In New Jersey," p. 57). 
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TABLE 6. POTENTIAL INDICATORS FOR OUTCOME GOAL IV FROM THE ORC SURVEY 



Among respondents whose age was 
16-29 years 30 years or more 



A. Producer roles: Percent who 
"frequently or regularly"; 

1. Put in extra hours at work 

2. Make a suggestion for improving 
pro^^dures at work 

3. Develop or learn new job skills 

4* Recommend their place of work to 
someone looking for a job similar 
to theirs 

5* Receive a job promotion 

Mean 



35 

27 
34 

24 
6 

25 



32 

30 
20 

13 
7 
20 



B* Consumer roles: Percent who 
"frequently or regularly": 

6* Compare prices when grocery shopping 42 

7* Compare the advantages of different 
brands or manufacturers when making 
a large purchase, such as a car, 
T.V*, or washing machine 50 

8* Consider the extra cost of borrowing 
money or using an installment plan when 
making a purchase 35 

9* Read any specialized consumer 

publication, such as Consiimer Reports 16 

Mean 36 



58 



55 

36 

20 
42 



Source: Computer printouts provided by Opinion Research Corporation 
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Table 5, concerning re-cycllng of materials, can be considered 
another Indicator of consumer resp mslblllty .) A question that might 
get at consumer satisfaction more clearly would be how often "yo" have 
made a major purchase that you were later sorry about." Unlike the 
other Indicators that have been discussed, goal attainment on this one 
would be measured by the decrease In the percentage saying that this 
happened "frequently 

Outcome Goal V 

The public schools of New Jersey should help every person 
In the state to acquire the ablllt^' to form satisfying and 
responsible relationships with a wide range of other people. 
Including but not limited to those with social and cultural 
characteristics different from his/her own. 

The principal thrust of this goal Is undoubtedly In the area of 
relationships between members of different racial groups. Consequently, 
one Indicator that Immediately suggests Itself Is the degree of resi- 
dential segregation of the races, on the premise that segregation decllne^s 
as people find greater satisfaction (or at least less dissatisfaction) 
In having persons of other races as their neighbors. Using the block 
data reported In the U.S. Census, Taeuber and Taeuber have developed 
an Index of residential segregation for cities which is essentially "the 
minimum percentage of nonwhites who would have to change the block 
on which they live in order to produce an unsegregated distribution — one 
in which the percentage of nonwhiten living on each block is the same 



24 



throughout the city" (Taeuber and Taeuber, 1965, p. 30). Thus, the 
value of the Index can vary from 0, which would be the case if the 
percentage of nonwhites were already the same on every block in the city, 
to 100, which would be the case if every block in the city contained 
only whites or only nonwhites. Taeuber and Taeuber have calculated 
the values of this index for 109 cities for which the necessary information 
was available in 1940, 1950, and 1960 and which had 1,000 or more dwelling 
units with a nonwhite head of household in 1940, and for another 98 
cities for which information was available only for 1960. 

Portions of their results, including all the New Jersey cities f'/r 
which the value of the index could be determined, are presented in 
Table 7. It shows that New Jersey cities were a little less segregated 
than those elsewhere, whether the comparison is with all cities or 
only with those in the Northeast or in the Middle Atlantic states; but 



Blacks constitute more than 90 percent of those classified by the 
Census as "nonwhite"; most of the rest are Oriental. Among those 
classified as "white" are, of course, many different ethnic groups, 
including those of Spanish-speaking background • 



Taeuber and Taeuber report values only for those cities for 
which block data were published by the Census. The same kind of index 
can be computed by using larger units, such as census tracts, though 
its values are not comparable to those based on blocks. 
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TABLE 7. INDICES OF RESIDENTIAL SEGREGATION BETWEEN 
WHITES AND NONWHITES, 1940, 1950, AND I960 









1940 


1950 


1960 


Mean for all 109 cities for which 
data were available 


85.2 


87.3 


86.1 


Mean for 25 cities in 


the 


Northeast 


83.2 


83.6 


78.9 


Mean for 18 cities in 
Atlantic states 


the 


Middle 


83.7 


84.3 


80.0 


New Jersey cities 












Atlantic City 






94.6 


94.0 


89.2 


Camden 






87.6 


89.6 


76.5 


East Orange 






85.3 


83.7 


. 71.2 


Elizabeth 






75.9 


76.1 


75.2 


Englewood 










87.9 


Jersey City 






79.5 


80.5 


77.9 


Montclair 










80.3 


Newark 






77.4 


76.9 


71.6 


Passaic 










71.8 


Paterson 






79.8 


80.0 


75.9 


Trenton 






81.9 


83.0 


79.6 



Mean for New Jersey cities 
Source: 



82.8 



83.0 



77.9 



Karl E. Taeuber and Alma F. Taeuber, Negroes in Cities; Residential 
Segragation and Neighborhood Change (Chicago: Aldine, 1965), pp. 32- 
34, 39-41, 44. 
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that they followed the same trend as other cities did — a slight 
increase in segregation from 1940 to 1950, and a small decrease 
between 1950 and 1960. It also shows that New Jersey cities, like 
those elsewhere, were far closer to the completely segregated than to 
the completely unsegregated value of the index, so there is plenty of 
room for movement toward the goal (which here again would be demonstrated 
by a decline in the indicator). However, it would probably be unrealistic 
to expect large changes to occur in the near future; between 1950 and 
1960, the average value for New Jersey cities dropped by only about 
half a point per year. 

An indicator of residential segregation is admittedly influenced 
by a major economic factor, the generally lower income of nonwhite families. 
One way of getting around this would be to develop a similar index of 
segregation in the schools. A community in which there were satisfying 
and responsible relationships between the races would presumably wish to 
desegregate its schools, and it could do so without regard to the incomes 
of students' families. There might be extra costs involved, as in 
transporting students, but the willingness to pay these costs would be 
a sign that the goal was being reached. 

More direct indicators of the ability alluded to in Outcome Goal V 
are found in the responses to several questions asked in the ORC survey. 
Some of these — see items 4-7 and, in part, item 3 of Table 8 — refer to 
differences other than racial ones. But the relationships called for 
in the goal are expressly "not limited to" people who differ from 
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TABLE 8. POTENTIAL INDICATORS FOR OUTCOME GOAL V FROM THE ORC SURVEY 

Among respondents whose age was 
16-29 7ears 30 years or more 



Percent who "frequeaLly or regularly" ; 

1. Come into contact with members of other 
racial groups In church, at meetings. In 
recreational activities, etc. 

2. Attend a social gathering where there 
are members of other races 

3. Speak uf for members of other racial 
or ethn'c groups 

4. Listen to someone who had different 
Ideas e7cplaln his Ideas to them 

5. Try to learn about people whose 
background and way of life Is 
different from their own 

6« Come into contact with people whose 
social and cultural characteristics 
are different from their own In church, 
at meetings, In recreational activities, 
etc. 

7. Attend a social gathering where there 
are people whose social and cultural 
characteristics are different from 
their own 

8. Lend something to a friend or 
neighbor 

9. Have friends over to their house 
or apartment 

10. Help a friend solve a problem 

11. Help someone who Is In trouble 

12. Welcome someone who has just moved 
Into the neighborhood 



34 
33 
31 
61 

35 



34 



ERiC 



Mean of items 1-3 
Mean of items 3-7 
Mean of items 8-12 
Source: Computer printouts provided by Opinion Research Corporation. 
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32 

67 

66 
59 
47 

22 
33 
39 
52 



26 



16 



20 



51 



24 



27 



17 

51 

55 
43 
43 

32 
21 
28 
45 



oneself, so it is especially noteworthy that some of the Items 
(8-12) refer to general social relationships apart from racial, social, 
or cultural differences. EGl's for this aspect of the goal would be 
very difficult to obtain except through a survey, and the figures in 
Table 8 suggest that progress toward its achievement is much greater 
than toward those aspects involving racial-, social, or cultural dif- 
ferences, so tb'^t it would be desirable to have separate measures. The 
higher percentages shown by the young on nearly every item are quite 
striking, though one must be cautious in attributing them to the recency 
of schooling alone; many other factors may be at work, including the 
often-noted cultural gap between the generations (which itself, however, 
may be partly due to education). Insofar as civic, work, and family 
responsibilities also involve relationships with other persons, some of 
the indicators for Outcome Goals III, IV, and VI could be used for Outcome 
Goal V as well. 

Outcome Goal VI 

The public schools of New Jersey should help every person 

in the state to acquire the capacities for playing satisfying 

and responsible roles in family life. 

An EGI that many people would think of in connection with this goal 
is the divorce rate — the higher the number of New Jersey citizens who 
are divorced (relative to some base number), the less often it would be 
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presumed they have learned to play "satisfying and responsible roles In 

family life." Actually, a divorce rate . In the strict sense of the 

term. Is not a useful Indicator for a state. In Its most acceptable 

form. It Is expressed as the ninnber of divorcers granted In a year relative 

to the number of ever-married couples for that year. While this rate Is 

meaningful for the country as a whole. It cannot be applied to a single 

state such as New Jersey because many, if not most, of its residents who 

12 

obtain divorces do so in other states. However, a divorce ratio 
could be calculated for a single state and would accomplish much the same 
purpose. This ratio is the number of divorced persons of either sex 
resident in the state in a given year relative tc the number of ever-married 
persons of that sex resident in the state in the same year. The ratios for 
New Jersey and the United States in 1950 and 1960 are given in Table 9. 

So many qualifications must be made about even these figures that r.heir 
usefulness as indicators is cast into serious doubt. Divorce is becoming 
increasingly acceptable to public opinion, so that (among other things) 
the legal restrictions on it are being relaxed; consequently, a rise in 



Data on the reported state of residence of t arsons seeking divorces 
in all states would not be helpful — even if they ^ available—since a 
person is generally required to establish a (temporary) residence in a 
state in order to qualify for a divorce there. 
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TABLE 9. DIVORCE RATIOS FOR THE UNITED STATES AND NEW JERSEY, 1950 AND 1960 



Males 

Number ever married 
(In thousands) 

Number divorced 
(In thousands) 

Divorce ratio (number 
divorced per thousand 
ever married) 



1950 

V.S. n.j. 

40201.3 1364.1 

1071.4 21.6 

26.6 15.9 



1960 
U.S. ■ N.J. 



45795.6 1641.7 
1289.1 25.5 



28.1 



15.5 



Females 

Member ever married 
(In thousands) 

Member divorced 
(In thousands) 

Divorce ratio (number 
divorced per thousand 
ever married) 



45684.0 1562.4 



1372.9 29.8 



30.0 19.1 



52439.4 1893.8 
1847.4 41.8 



35.2 



22.1 



Source: U.S. Bureau of the Census, U.S. Census of Population: 1960. 

Detailed Characteristics. United States Summary. Final Report 
PC (I)-ID (Washington: Government Printing Office, 1963), p. 436; 
U.S. Bureau of the Census, U.S. Census of Population: 1960. 
Detailed Characteristics. New Jersey. Final Report PC (1)-32D 
(Washington: Government Printing Office, 1962), pp. 381, 383. 
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the number of divorces may occur without signifying a corresponding 

rise in the number of marriages Involving dissatisfied or Irresponsible 

spouses. Indeed, divorce may Itself be a highly responsible act in some 

circumstances— for example, where it is to the benefit of the children of 

an unhappy marriage; to the extent that this is true, a rising divorce 

ratio could even indicate more rather than less responsible performance 
13 

of family roles. At the same time, there are many people, especially 
in some religious groups, for whom divorce continues to be unacceptable 
and who would not resort to it even if they felt their spouse was 
irresponsible or they were dissatisfied with their marriage for some other 
reason. Nor does the indicator take any account of single persons, some 
of whom may be single precisely because they have characteristics which 
have prevented them from becoming responsible partners in a satisfying 
marriage. 

There is another objection which is specific to the use of this indicator 
for purposes of measuring the attainment of educational goals. In most of 
the goal areas, education can properly be regarded as a life-long process. 



There are also complicated problems arising from the frequency of, 
and trends in, re-^rriage after divorce. Divorced persons who re-marry — 
and the great majority of them do — occupy an ambiguous position with 
respect to Outcome Goal VI. If they are re-married at the time of 
enumeration, they are of course Included in the number of married persons 
but not in the number of divorced persons. 
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and the state's educational system can be held accountable for remedy- 
ing deficiencies In elementary or secondary schooling, wherever It took 
place, by providing adult education. But divorce Is an event that, 
having occurred, cannot be "remedied" by later education. The divorce 
ratio therefore cannot be Interpreted as an EGI for New Jersey without 
knowing how many people have moved Into the state after having been 
divorced (and educated) elsewhere, and such Information Is not available. 
Conversely, there Is no way of knowing how many people have been educated 
In New Jersey and have been divorced but now reside In some other state. 
It would be hazardous to assume that this In-onlgratlon and out -migration 
are neatly balanced. 

Another potential EGI for Outcome Goal VI, emphasizing the "responsi- 
bility" part of It, Is the rate of Illegitimate births (number of Illegitimate 

14 

births In a year per 1,000 live births In that year). Data for New Jersey 
and the United States are presented In Table 10. An especially noteworthy 
point about this table Is the sharp rise In the ratio that occurred between 
1960 and 1966, both In the state and In the nation as a whole. So great Is 



Some current conmientary would suggest that a reduction In the birth 
rate Itself — the number of live births per year per 1,000 women of child- 
bearing age — Is a sign of Increasing parental responsibility. The 
controversy that would be aroused by the proposal to use this rate as an 
EGI Is obvious. 
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TABLE 10. RATES OF ILLEGITIMATE BIRTHS IN NEW JERSEY 



AND THE UNITED 


STATES BY 


AGE, 1954, 


1960, 


AND 1966 








1954 


1960 




1966 




Number of Illegitimate births 
per 1,000 live births among 
women whose age was: 


U.S. 


N.J. 


U.S. 


N.J. 


U.S. 


N.J. 


Less than 15 


539 


841 


530 


764 


763 


910 


15-19 


114 


154 


110 


164 


219 


272 


20-24 


33 


31 


34 


41 


71 


69 


25-29 


18 


10 


21 


17 


41 


30 


30-34 


16 


8 


19 


12 


38 


24 


35 or more 


18 


9 


21 


12 


42 


20 


Total, all ages 


34 


24 


38 


36 


84 


72 
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this Increase that it is almost certainly due not to any actual change 
In the ratio but rather to something liavlng to do with the reporting 
of the number of Illegitimate births. It Is true that the number of 
Illegitimate births reported has risen with the Increasing proportion of 
births which take place In hospitals which Is just (one of the extraneous 
factors affecting the ratio), but the magnitude of this change Is too 
great to be accounted for by that. Efforts to ascertain what else may 
have been Involved were unsuccessful, so the table must stand as a 
warning against hasty Interpretation of such figures and particularly 
against attributing changes In them to changes In the educational system. 

ORG survey Indicators for this goal are listed In Table 11. What 
Is most attractive about these Items Is that they are equally applicable 
to any family member, regardless of sex or age (except for the very 
youngest, who are not relevant to EGl's, anyway). 

Outcome Goal VII 

The public schools of New Jersey should help every person 
In the state to acquire the knowledge, habits, and attitudes 
that promote personal and public health, both physical and 
emotional. 

In the case of health, as In that of emplo3anent, a wealth of 
statistics conceals a paucity of measures that would be useful for EGl's. 
There Is no space here to go into all the complex reasons for this (for an 
excellent discussion, see Morlyama, 1968), but briefly, the Indexes of 



It should also be pointed out that the high ratios In the under-15 
age bracket are due not to a large number of Illegitimate births*-- the 
number Is actually much smaller than In any other age group**-but rather 
to the Infrequency of marriage among women of that age. 
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TABLE 11. POTENTIAL INDICATORS FOR OUTCOME GOAL VI FROM THE ORC SURVEY 



Among respondents whose age was: 
16-29 years 30 years or more 



Percent who "frequently or regularly": 



Celebrate an anniversary or 
birthday with a member of their 

family 72 77 

Help a member of their family 

solve a problem 52 63 

Make plans to do something with 

other members of their family 50 65 

Keep in touch with a relative 

who lives away from home 45 63 

Do something their family wants 
them to do which they wouldn't 

otherwise do 40 38 



Mear. 52 61 
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mortality, life expectancy. Illness, and Injury are such that they are 
very unlikely to be noticeably affected by improvements In the quality 
of education. Any changes that do take place are far more likely to 
be the results of changes in medical knowledge and technology, hospital 
facilities, and the cost and accessibility of medical care, or even in 
housing, or in transportation and communication (which determine the 
rapidity with which medical help can be obtained).''^ 

It might appear that an important possible exception to this 
generalization would be venereal disease rates. This form of illness 
is especially prevalent among the young, and (unlike m^^y other coimnon 
diseases) its rate probably could be substantially reduced by appropriate 
education. Ironically, however, the principal effect of education for 
some years might well be to encourage the reporting of cases — so that 
they could be treated — and thus to produce an apparent rise in the rate, 
even though the actual rate might be going down. 

Traffic accidents, too, may be relatively easily influenced by educational 
efforts (although they are also affected by technology and practice in 
highway and automobile construction and traffic control), and they are 
probably comprehensively reported, particularly when injuries are incurred. 
The problem with such an indicator is that a considerable proportion of 
accidents in New Jersey must involve out-of-state drivers, and the 
proportion varies over time (and among different states). Accidents reports 
do not at present make this distinction. 



An increased amount of education may bring a person better health, 
by enabling him to qualify for a higher-income job, with all that that implies 
(Lee, 1969, pp. 444-446). But EGl's are supposed to measure increases in 
the quality, not quantity, of education. 
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The situation Is much worse as far as "emotional" or mental health 
Is concerned* There Is not even agreement about what constltues mental 
health or Illness* A recent survey of Indicators concluded, for this 
area: 

Because of still unresolved problems of psychiatric diagnosis, and 
because the types of beha\-.or that are considered manifestations 
of mental Illness change with our culture, no adequate measures of 
the mental health of a population have been developed. (U.S. Depart- 
ment of Health, Education, and Welfare, [1969], p. A) 

Furthermore, If an Indicator were to be devised, a major difficulty In 

using It as an EGI would arise from the stigma often attached to mental 

Illness, which deters Its victims or their relatives from seeking help 

or even acknowledging the condition. Perhaps the most constructive action 

that could be taken by the educational system, therefore, would be to 

change the attitudes that lead to stlgmatlzatlon— but that would again 

(as with venereal disease, and for similar reasons) lead to an apparent 

increase in the rate, at lea in the short run. 

Turning from peisonal to public healtL, one potential and available 

measure is the proportion of the population served by a fluoridated water 

supply. The figure was 14.4 percent for New Jersey in 1970 (including both 

natural and controlled fluoridation), wMch placed it 43rd among the states. 

Data obtained from the Office of the Bureau of Dental Health, New Jersey 

Department of Health. This could be taken as an indicator of "attitudes 

that promote" public health — but it hardly needs to be pointed out that 

its validity would be challenged by many people. Measures of the frequency 

or magnitude of efforts to eliminate various forms of environmental pollution 

might serve the same purpose with less controversy. 
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The results of a number of health-relevant questions Included in 
the ORG survey are given in Table 12. Items 1-6 are positive indicators; 
they show, not surprisingly, that the old tend to take more health 
precautions than the young. Items 10-12 are also positive; they asked 
respondents to say whether each of the three menus for a mea? was "balanced," 
the menus having been selected by experts for a similar pool conducted 
for CBS News. Of the nationally representtative sample queried in that 
poll, an average of 73 percent responded correctly to the thtee menus — 
slightly better than the New Jersey sample did.^"^ 

Item 7-9 were intended to be negative indicators — they would show 
goal attainment by lower rather than higher values — but they are somewhat 
problematical. It is hard to know, for exain*^le, what a rise in the 
"frequently" response to item 8 would mean: that respondents are more 
frequently sick (and that the health of the population is therefore 
declining), or that when respondents are sick, they more frequently stay 
in bed (and that health habits are therefore improving) . One may also 
question the reliability of a question about the use of hard drugs — 
although it must be said that other sources of information, such as police 
or hospital records, would not inspire any greater confidence. Collectively, 
the three items vividly demonstrate that a simple average of responses 
may not be justified, even if the response format is the same, because the 
attitudes or behaviors may be qualitatively different in importance. To 



"Goals for Elementary and Secondary Schools for New Jersey," p. 107. 
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TABLE 12. POTENTIAL INDICATORS FOR OUTCOME GOAL VII FROM THE ORC SURVEY 



\mong respondents whose age was: 
16-29 years 30 years or more 



Percent who "frequently or regularly"; 

1. Stay within the speed limit 
when driving a car 

2. Stay home or keep children at 
home to keep a cold or the flu 
from spreading to other people 

3. Use products that help control 
pollution, such as low-lead 
gasoline or special detergent 

4. Go to a doctor for a check-up 

5. Go to a dentist for a check-up 

6. Get physical exercise 

7. Smoke a cigarette 

8. Are sick enough to stay in bed 

9. Use hard drugs, such as heroin 



41 
41 

39 
35 
38 
53 
37 
6 



68 

56 

45 
49 

43 
37 
32 
6 



Percent who correctly distinguished 
between balanced and Imbalance meals ; 

10. Menu A 82 78 

11. Menu B 71 69 

12. Menu C 53 56 

Mean of items 1-6 41 50 

Mean of items 7-9 14 13 

Mean of items 10-12 69 68 



*Less than 0.5 percent. 

Source; Computer printouts provided by Opinion Research Corporation. 
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cope with that problem, the items could be assigned different weights, 
derived from a consensus of either expert or public opinion. 

Goal Outcome VIII 

The public schools of New Jersey should help every person 
In the state to acquire the ability and the desire to 
express himself /herself creatively In one or more of the 
arts, and to appreciate the esthetic expressions of other 
people* 

The search for Indicators In this area was one of the least 
productive. Five Items that were Included In the ORG survey are shown 
In Table 13. They could be supplemented by others, such as frequency 
of reading a novel, attending a film, or subscribing to a music or art 
magazine. One serious problem here Is whether the quality as well as the 
quantity of esthetic activities should be taken Into account — whether, 
for example, watching a Bergman film Is to be equated with watching a horror 
production. Related to this is the difficulty of making a distinction 
between esthetics and entertainment. 

Outside of the survey items, tha nearest thing to an indicator that 
could be located was a list, provided by the New Jersey State Arts Council, 
of "cultural" organizations active in the state la 1967. It concained the 
names of 37 orchestras, 16 choral and dance groups, 15 opera associations, 
8 museums, 6 theaters, and 5 "arts centers." No information was available 
on their degree of activity, the number of their participants, or the size 
of their audiences. 

The scarcity of data on this important educational and human domain 
is not peculiar to New Jersey. According to Toffler, who has given close 
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TABLE 13. POTENTIAL INDICATORS FOR OUTCOME GOAL VIII FROM ORC SURVEY 



Percent who "frequently or regularly"; 



Among respondents whose age was: 
16-29 years 30 years or more 



1* Use their creative abilities in 
handicrafts, such as sewing, 
knitting, woodworking, etc. 

2. Use their creative abilities in 
such things as painting a picture, 
writing a story or poem, playing 
a musical instrument 



41 



32 



38 



15 



3* Sing in a chorus, perform in a 
play or concert, etc* 

4. Attend a play, concert, art 
exhibit, or other cultural event 

5. Go to a museum or art gallery 



12 

27 
13 



18 
10 



Mean of items 1-5 



25 



17 



Source: Printouts provided by Opinion Research Corporation. 
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consideration to measurement of the arts, "it is safe to say that in 
no other significant field is the existing data base flimsier" (1969, 
p. 267). Indeed, as he also points out, "The very idea of measuring the 
arts is abhorrent to many" (Toffler, 1969, p. 263)* He has himself made 
a bold and provocative set of proposals for the establishment of a 
"cultural data system" (Toffler, 1969, pp. 268-277), but they are full 
of gross defects and probably impossible of realization. 

Outcome Goal IX 

The public schools of New Jersey should help every person 

in the state to acquire an understanding of ethical principles 

and values and the ability to apply them to his/her own life. 

It may well be that attainment in this goal area defies measurement 
altogether, at least without a clearer specification of exactly which 
"ethical principles and values" are supposed to be taught in New Jersey's 
schools. Making some assumptions about that, some of the ORG survey items 
might qualify as EGl's fo; Outcome Goal IX — for example, those which seek 
to tap "responsible" attitudes and behavior in civic, work, and family 
roles, in relationships with other people, and in health. 

The vast domain of crime statistics was investigated for possible EGl's 
of indicators of ethics, but their shortcomings give one pause. Most such 
statistics are based on the number of crimes reported ^o the police, and 
these are but a small proportion of the number of crimes committed. 
Consequently, the rates are easily influenced by anything that induces people 
to report crimes more often than they do now— a well-publicized anti-crime 
drive, or an increase in police visibility or in the number of crime stories 
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appearing In the press. Many other extraneous matters enter into 
the measurements; for example, the single most Important factor responsible 
for the reduction of the homicide rate In the past 50 years has been the 
Improvement In medical care, which has prevented many murderous assaults 
from resulting In death. This, In turn, raises the question of just 
which crimes are appropriate Indicators of lack of ethical principles; 
not only do categories of crime vary In serli>usness from parking violations 
to homicide and forcible rape, but there Is a great deal of variation within 
a category as well. (For further discussion of these and related problems, 
see Glaser, 1969, and Sellln and Wolfgang, 196A.) Perhaps the most signifi- 
cant question Is whether crime Is to be considered In the context of 
ethical principles at all, or In the context of vocational skills or 
family life or relationships with others — If, Indeed, it is to be assuned 
that educational change per se . In the absence of social and economic changes 
of a much broader kind, could have any effect on crime rates. The etiology 



This problem could be avoided by the use of a rate of crime 
victimization Instead of a rate of. crime commission. It would be a survey- 
type Indicator, based on the number of times a sample of respondents say 
they have been the objects of criminal acts. A survey of this kind has' 
been carried out on a sample of the United States population (Glaser, 
1969, p. 419), and the relevant Items could be Included In future surveys 
conducted for purposes of collecting EGI data In New Jersey. One limitation 
of this form of measurement, as Glaser mentions, is that it does not 
include many crimes of property against private and governmental organizations. 
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of crime Is an exceedingly complex Issue* For whatever It is worth, 
there were 236.1 reported crimes of violence per 100,000 residents in 
New Jersey in 1968, and 2,201.5 reported crimes against property; the 
corresponding figures for the entire United States in the same year were 
294.6 and 1,940.2 (U.S. Department of Labor, 1970, p. 58). 

Outcome Goal X 



The public schools of New Jersey should help every person in the 
state to develop an understanding of his/her own worth, abilities, 
potentialities, and limitations. 

None of the ORC sur/ey items were pertinent to this goal, nor were any 
data located for other ;mitable indicators of it. Indeed, this may well be 
the most difflculL to measure of all the goals. 

Assuming that development of an understanding of one's own worth 
means that people should come to think well of themselves, to have self- 
respect or self-esteem, attainment of this part of the goal would be 
indicated by the prevalence of "positive self -concepts" in the population. 
But the concept of "self-concept" is extraordinarily elusive. There is not 
even agreement on definition of the term. As one student of the subject has 
said, "The research is relatively limited, standardized measures are largely 
nonexistent, and terminology is obscure" (Bourisseau, 1972, p. 81; for 
an earlier but more comprehensive review, see Wylie, 1961). Thus, it is 
probably unreasonable to expect that widely accepted measures of the under- 
standing of one's own worth could be proposed at the present time. 
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Though the terms seem more clear-cut, measurement of the under- 
standing of one's "abilities, potentialities, and limitations" is not 
much easier. A basic problem is knowing exactly what is intended by 
the word "understanding." It may mean nothing more than "knowledge" 
of the stated qualities, or it may be taken more broadly to mean "appreciation 
of their significance." The latter Interpretation presents a dilemma: 
To accept a person* s own opinion about the significance of his characteristics 
would be tautologous (everyone would then always understand his own 
abilities, by definition, and the measure could show no variation over 
time); yet to accept someone else's judgment of their significance, however 
that might be arrived at, would entail the assumption that a person 
appreciates the significance of his own qualities, and in that sense 
"understands" himself, less well than others do — an assumption that one 
might properly be reluctant to make. 

Equally awkward problems are raised if understanding is to mean 

only knowledge, thus implying that there are more nearly "objective" facts 

about himself that a person can and should know. There are, of course, 

many instruments for making objective assessments of an individual's 

abilities, potentialities, and limitations, and respondents in a survey 

19 

might be asked whether they knew their scores on various such tests. 



The responses would reflect such extraneous matters as the frequency 
with which tests are given—which is rising— as well as varying practices 
in informing test-takers of their scores. For interesting evidence and 
observations on these and other issues connected <ith the measurement of this 
goal, see Brim and others, 1969, especially pp. T 9-269. 
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There is no assurance, howev/:r, that their professed knowledge would 
be accurate. To get around this, a procedure such as this might be 
employed: Respondents could be told that they were going to be given, 
say, a 20-item test of college-level vocabulary and asked how many 
items they believed they would answer correctly; then the test would be 
administered, and the respondent's knowledge of this ability would be 
measured by the difference between his estimated and actual scores. There 
are, however, a number of technical difficulties in this procedure. 
Furthermore, the results would be limited to a single ability, and it would 
probably be too cumbersome to follow a similar procedure for any large 
number of abilities. There are some abilities — not to mention potenti- 
alities and limitations — that could not be measured at all in this way. 

An alternative strategy would be to ask a sample of the population 
simply whether they thought that their abilities were "average," "below 
average," or "above average" in comparison with some reference group. 
The abilities could be differentiated into various kinds, and the reference 
groups might include "the people with whom you work," "most of your friends, 
"or" the average person In this state" or "in the United States." 
The presumption here would be that, since most abilities are distributed 
in a statistically normal fashion, attainment of the goal of "knowledge 
of abilities" would mean that most people will say they are average and 
that the proportion who say they are above average will be approximately 
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equal to the proportion who say they are below average. This 
eliminates the necessity of determining whether any Individual's 
knowledge of his abilities Is correct, but It leads to a peculiar 
contradiction that Illustrates the complexities Involved In measurement 
of this goal. Saying that one's abilities are above average Is probably 
a sign of positive self-concept. An increase in the proportion who give 
this answer could therefore be taken both as evidence that the goal of 
understanding one's worth was being achieved and as evidence that the goal 
of understanding one's abilities was not being achieved! 

Outcome Goal XI 

The public schools of New Jersey should help every person in 
the state to learn to enjoy the process of learning and to 
acquire the skills necessary for a lifetime of continuous 
learning and adaptation to change. 

People who enjoy the process of learning and have the skills for it 
presumably engage in it as a voluntary action more frequently than those 



When a similar question was asked about intelligence in a recent 
national survey of adults, between 68 and 85 percent of the respondents 
said they were "the same" as others, depending on the reference group; 
the proportions saying they were "higher" ranged from 12 to 30 percent 
and were three to fifteen times as great as the proportions saying 
they were "lower" (Brim and others, 1969, p, 261). 
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who do not enjoy It. Consequently, the proportion of the population 

Involved In formal but noncompulsory education might be an EGI for this 

goal. Slightly more than two-thirds (67 percent) of those who graduated 

from New Jersey's high schools during the 1968-69 school year were 

reported to have embarked upon some form of postsecondary formal 

21 

education the following fall. The great majority of them — almost 85 
percent — entered a two-year or four-year college or university, but 
It would be Important not to limit the Indicator to that group. In order 
to avoid the Implication that college Is the proper form of postsecondary 
education for all. (Of course. It ought not be assumed that continuing 
one's education In any form Immediately upon graduation from high school 
Is necessarily the "right" thing for everybody, which suggests, as has 
been pointed out before, that no single Indicator Is adequate for 
measuring the attainment of a goal.) This Indicator does not take 
account of those who left high school before graduation; and, as usual. It 
Is subject to extraneous Influences — most particularly. In this case, the 
cost, accessibility, and admissions policies of postsecondary Institutions. 
Some doubt may also be expressed about the "voluntary" nature of 
postsecondary education. How many students go to college because of 
parental pressures or felt vocational needs? While people who enjoy 
learning may engage In It more often than those who do not enjoy It, by 



^ ^New Jersey Education , 4 (April, 1970), 4-5. The figure Is derived 
from forms completed by high-school principals on the basis of probably 
Incomplete Knowledge* 
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no means all who are engaged in it are enjoying It* 

Another type o£ formal noncompulsory learning experience are 
the adult education courses offered by most school districts. In 

22 

1967-68, there were 185,196 enrollments ir. these courses In New Jersey, 

or 393 per 10,000 adult (18 years old or more) residents of the state. 

Here, too, there may be some question as to whether these enrollments 

represent learning for the sake of enjoyment alone — many courses, for 

23 

example, are clearly oriented toward vocational Improvement — but that 
oay be an unnecessarily rigid distinction to make In the case of this 
kind of case. However, course-by-course figures are avallable^i and 
classifications by type could be made If that were desired. 

Indicators for more Informal sorts of learning experiences are 
Considerably less promising* One might suggest the circulation of books 
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Data provided by the Bureau of Adult Education of the New Jersey 
State Department of Education. 



23 

In this connection, it should be mentioned that many business 
firms offer more or less formal training to their employees. No compilation 
of the numbers participating could be found, but it should not ie 
difficult to collect such data. Note that there is an ORG survey item 
asking about the learning of new job skills (Table 6, item 3). 
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by public libraries (nearly 29,000,000 for New Jersey's 355 public 
libraries In 1968 ), but school-age children probably contribute 
heavily to that figure; there are different definitions of "circulation"; 
and there Is no assurance that a book borrowed Is a book read (much 
less learned from or enjoyed). There are similar difficulties, and 
others as well, with such data as museum attendance and newspaper 
circulation rates. 

Both formal and Informal learning experiences \i;jre touched on by 
ORC survey Items, which are, moreover, somewhat freer of vocational 
considerations. One Interesting aspect of the responses (Table lA ) 
Is that older people seem to prefer briefer or less "demanding" sources 
of Information and knowledge, such as newspapers and television, while 
younger people— perhaps because they still retain the habits of formal 
schooling — are apparently more willing to read books and attend courses 
or lectures. When all the types of learning experiences asked about are 
weighted equally, however, there is no significant difference by age. 
These items are not direct evidence of enjoyment any more than the other 
kinds of indicators are; but where a variety of voluntary activities can 
be considered together and on the same scale, the principle, that higher 
frequency implies greater enjoyment may be reasonably valid. 



New Jersey State Library, Public and School Library Service Bureau, 
Public Libraries of New Jersey^ Statistics 1968 (Trenton: State Department 
of Education, 1969, pp. 30,31. 
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None of the potential Indicators that have been discussed for 
Outcome Goal XI Is an adequate measure of whether people have the 
skills r4ecessary for "adaptation to change**' That Is a peculiarly 
ambiguous part of the goal, though* Ar^ New Jersey's citizens to be 
expected to adapt equally well to all changes — to new production 
technologies and Increased pollution, to population movements and 
Intensified racial conflict, to political campaigning by television and 
greater restrictions on free speech? Until there Is clarification of 
what kinds of changes are supposed to be adapted to and what kinds 
resisted — or, to put It differently, what kinds of behavior "adaptation" 
Is Intended to mean In what kinds of circumstances — measurement of this 
part of the goal does not seem possible* 
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TABLE 14. POTENTIAL INDICATORS FOR OUTCOME GOAL XI FROM ORC SURVEY 



Among respondents whose age was : 
16-29 years 30 years or more 



Percent who "frequently or regularly": 

1. Spend at least 20 minutes reading 

a newspaper or newspapers 56 77 

2* Keep informed about current events 54 65 

3* Read magazines or journals to get 
information, not just for entertain- 
ment 55 57 

4* Watch T«V* to get information, not 

just for entertainment 45 60 

5. Read books 58 49 

6* Expand their knowledge of a subject 

by talking with other people 57 46 

7* Attend a course or a lecture to further 

their knowledge 29 14 



Mean 51 53 



Source: Computer printouts provided by Opinion Research Corporation. 
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Conclusions 



The findings of this search for educational goal Indicators suggest 
that a distinction should be made between two types of Indicators, which 
may be called Intentional and Incidental , Intentional Indicators are 
those based on data gathered expressly for the purpose of measuring the 
degree of attainment of educational goals; Incidental Indicators are 
those based on data gathered for some other purpose. In the context of 
this report. Intentional Indicators are synonymous with the ORG survey Items 
and Incidental Indicators with all the rest (although, for reasons to be 
explained below, they should not be called "survey" and "non-survey" 
Indicators) . 

By compelling the consideration of specific potential Indicators for 
New Jersey's educational goals, the search has also helped to reveal the 
characteristics of good EGl's. In retrospect. It appears that there are 
ten such characteristics. A comparison of the advantages and disadvantages 
of intentional and Incidental Indicators In their terms will serve to 
summarize what has been found and should point toward appropriate actions for 
Improving measurement of the attainment of educational goals. 

A good EGI has an Intrinsically meaningful significance . 
This Is part of the very definition of an Indicator, and properly 
selected Indicators of both types exhibit It equally well. 

2- A good EGI retains a constant meaning over a relatively long period of time 
and when changing conditions do require a < liange In Its f orm> such 
change Is accomplished readlly> without drastic alteration of meaning . 
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Repeatedly, it has been found that the relationship of Incidental 
Indicators to education outcomes Is subject to adrupt change brought 
about by a new product, or a new lav, or the more (or less) rigorous 
enforcement of an old lav. When that happens, the Indicators can no 
longer be used to measure educational progress (though they may still be 
useful for other purposes, to vhlch they may not be "Incidental" at all) • 
Such changes are beyond the control of the educational system. Intentional 
Indicators, on the other hand, can be deliberately designed so as to be 
broad or generalized enough to reflect an educationally relevant activity 
in a fairly constant vay even vhen the circumstances surrounding that 
activity change. Eventually, they, too, may be made obsolete by changing 
conditions; but even then, if they are based on survey questions, they 
can be more easily cast into nevly useful forms than is the care vhere 
data collection depends on a complex institutional reporting netvork, as is 
true of many incidental indicators. 

3. A good EGI is sensitive to vhat is happening in education and insensitive 
to vhat is happening outside it . 

Failure to meet this criterion is one of the major dravbacks of incidental 

indicanors. Almost every one is affected extraneous, non-educational forces 

that may ver> veil blot out the direct effects of education (Sheldon and 

Freeman, 1970, pp. 100-101). Intentional indicators are not free of this 

difficulty; hovever, because they are specifically designed to measure 

educational effects, they can come closer to avoiding it. (See also the 

discussion of characteristic 6.) 
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4, A good EGI Is part of a larger body of data that allows the 

determination of what a reasonable level of attainment would be for 
New Jersey , 

At first glance, this criterion would seem to mean that there oaouid 
be comparable data for the United States or for other states or regions 
separately. If so, incidental indicators would be preferable, since 
many of them are derived from data gathered for the entire nation. Most 
of the ORC items were asked for the first time in this survey; there is 
no reason in principle why they could not be asked elsewhere as well, 
perhaps by other states for the same purpose, but that would require time 
and coordinated effort, with uncertain results. 

However, a word of caution is in order. Although there has not been 
space to explore the issue fully in this report, the territorial units with 
which New Jersey should be conq>ared are by no means self-evident. The 
attainment of educational goals is inevitably conditioned by non-educational 
factors, such as degree of urbanization, socioeconomic composition, patterns 
of transportation and communication, etc. These conditions are not 
necessarily the same in the nation as a whole, or in particular other 
states, as they are in this state. Hence, the levels attained elsewhere 
may be higher, or lower, than those that are reasonable for New Jersey. 
Comparisons should therefore be made with "comparable" states or regions— 
but it is not easy to decide which ones those are. The decision may vary 
from goal to goal, and then the data for the appropriate units may not be 
available for some incidental indicators. Indeed, in some instances, it 
may be possible, or even preferable, to set a level of goal attainment in 
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terms of internal variation within the state — e.g., "the state as a 
whole should reach, in five years, the level of the county presently 
at the first quartile." If educational objectives are defined in this 
way, intentional indicators would generally be superior, since they 
could be obtained for any desired unit within the state, while the data 
for incidental indicators are often not available for units smaller 
than the entire state. 

5. A good EGI is one among a set of similar EGl's that allow a reliable 
estimate to be made of the trend with respect to each goal and that 
collectively cover all goals . 

Because they were devised for a wide variety of purposes, incidental 

indicators are apt to be so heterogeneous in form as to make it impossible 

to combine them in any meaningful way; thus, one indicator may Imply 

that the schools are moving in one direction while another indicator suggest 

the opposite, and there is no way of knowing which one is more nearly 

correct or of "averaging" their values. Moreover, there arc some 

educationally important domains in which, for various reasons (including 

the absence of any other institution with sufficiently broad responsibilitie 

or with any responsibility at all), no indicators incidental to education's 

purposes are in use, or not enough to allow reliable judgments of progress 

to be made. By contrast, intentional Indicators are not constrained by 

the needs or lack of needs of other institutions; there is virtually no 

limit to the number of them that can be developed for a given goal; and 

they can be designed in compatible forms that permit summation and the 
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calculation of a central tendency for each goal. It Is worth noting 
that these latter two attributes give rise to another advantage: 
If the number of Indicators Is large, and If they tap different aspects 
of the same goal, there Is a greater probability that the effects of 
extraneous forces will be canceled out, and the measure of central 
tendency will then be a better reflection of educational activities. 

There may be some goals for which satisfactory Intentional 
Indicators cannot presently be devised > either—although In some cases 
the problem seems to lie In the ambiguity of the goal statements them- 
selves. Nevertheless, Intentional Indicators do promise more nearly 
adequate coverage of each goal and of all goals together than Incidental 
Indicators do. 

6» A good EGI provides an accurate measure of the goal status of the 
entire population and of subgroups of special concern . 

By virtue of the purposes to which they pertain, Incidental Indicators 

are often based on the performance or actions of more or less self-selected 

groups— those who apply for college or who marry or who seek a divorce, 

for example. Consequently, they give a biased picture of the total 

population. It may be precisely those \AiO do not select themselves Into 

these groups whose needs the schools are most seriously falling to meet In 

the relevant respect. Furthermore, some racial and ethnic groups are 

so small a proportion of the population that they are sometimes present 

In Incidental Indicator data in too few numbers to permit a reliable judgment 

of the trends among them; yet they, too, may be groups for whom the 

shortcomings of the educational system are especially great. Black and 



58 



Spanish-Speaking minorities are obvious cases in point. Intentional 
indicators, on the other hand, can be based both on representative 
samples of the whole population and on over-samples of small groups that 
will ensure their representation in sufficient numbers (with suitable 
weights then applied to their responses so that they do not unduly 
affect 'he picture for the total population)* 

7. A good EGI enables distinctions to be made among individuals with 
di fferent attributes^ and especially between those who have received 
their education In New Jersey and those who have not > 

An EGI is the more helpful as it permits the more precise identification 

of those groups in which educational progress or lack of progress is most 

conspicuous* Many incidental Indicators are, or can be, categorized in 

terms of age, sex, racial/ethnic membership, urban/suburban/rural residence, 

etc* Indeed, there are more possloillties for such categorizations 

than this report has been able to demonstrate* The number and type of 

categories, however, varies from one indicator to another, and for some 

the appropriate types are not available at all. In particular, no 

incidental indicator distinguishes between persons who have been educated in 

New Jersey and those who have been educated elsewhere; tliis distinction is 

simply irrelevant to the purposes of health care, or law enforcement, or 

any of the other purposes for which they have been developed* But the 

distinction is of course crucial to judgments of the performance of the 

staters educational system. With intentional indicators in the form of 

survey questions, it Is a simple matter to make the distinction with a 
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question about where the respondent went to school- At the same 
time, the use of other questions would permit categorizations also to be 
made In any of the ways employed for Incidental indicators* 

8» A good EGI has a moderate value among at least most groups of the 
population . 

That is. It should not already be at so high a level as to preclude 
the observation of progress nor at so low a level as to preclude the 
observation of decline* Intentional indicators again have the advantage 
here, since this characteristic can be deliberately built into them, 
whereas incidental Indicators have to be taken at whatever values they 
have. 

9- A good EGI is not subject to distortion by individuals or groups with 
an interest in maximialng or minimizing the appearance of educational 
progress . 

An ever-present danger of measurement is the reactive effect — the 
alteration of data caused by the very process of collecting them. 
Intentional Indicators appear to be especially vulnerable to this danger; 
the person who is asked a question can "choose" his answer, consciously or 
unconsciously, so as to produce some desired result. Thus, someone with 
a "grudge" against the school he attended, when presented with a series 
of items that he knows are going to be used to measure educational outcomes. 



The ORG survey asked respondents how many years they had attended 
public schools in New Jersey, and the survey report divides them into 
those whose answer was six years or more and those whose answer was less 
than six years. In a future survey, the distinction could be made at 
another point or points, as desired, so long as the numbers in each category 
were large enough to permit statistically reliable calculations to be made. 
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might answer In such a way as to make the schools "look bad." The 
opposltiB might be done by a teacher who felt that the Indicators amounted 
to Pleasures of his performance, or by a taxpayer who feared that low 
Indicator levels might lead to a demand for more money for the schools. 
These reactive effects seem less likely to occur with Incidental 
Indicators, which by definition are derived from data not collected for 
purposes of measuring educational progress. Indeed, some Incidental 
Indicators are altogether "unobtrusive" (Webb and others, 1966), In 
the sense that the relevant data are Inherent In the behavior they 
measure rather than being the result of any special effort at data- 
gathering. The proportion of the eligible population that Is registered 
to vote Is an Illustration of this quality. 

Ideally, then, an Indicator should be unobtrusive or at least non- 
reactive, and Incidental Indicators more often meet these desiderate 
than Intentional Indicators do. But the differences are not as great as 
they might appear to be. In the first place, while Incidental Indicators 
may not be affected by concerns for what they show about educational 
Institutions, they may very well be affected by concerns for what they show 
about the Institution whose performance they more directly reflect. Crime 
rates, for example, say something about law enforcement; and since the data 
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The widespread existence of such "grudges" would Itself be an 
Indicator of the workings of the educational system! However, It would 
be Impossible to distinguish those Item responses that reflected this sort 
of diffuse dissatisfaction from those that Indicated a genuinely low level 
of attainment on the goal la question. 
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are generally collected and published by law-enforcement authorities, 

distortions of various kinds creep in. Secondly, some measures that 

have been suggested for EGI use are themselves based on survey 

questions (which is why intentional and incidental indicators have not 

been referred to as "survey" and "non-survey" indicators); they iiave the 

appearance of non-reactiveness only because the survey was conducted 

prior to the time the data were put to EGI use. As this use became a 

27 

matter of public knowledge, they would no longer be non-reactive. 
Finally, the tendency of respondents to manipulate their answers to survey 
questions should not be exaggerated. Long experience with suirvrys has 
shown that it does not actually happen on a large scale (perhaps partly 
because, in addition to their immediate "selfish" interests, people also 
have an interest in helping to produce accurate information about public 
affairs), and a great deal is now known about how to phrase questions so as 
to minimize it. 

10. A good EGI is based upon data that are inexpensive to collect . 

This may be one of the most attractive-seeming advantages of 
incidental indicators. The collection of information can be quite costly. 



It is often overlooked, for example, that Census statistics, the 
source of many incidental indicators, are actually based on survey questions. 
It is true that there is an important deterrent to manipulation of the answers 
to Census questions — willfully false answers carry criminal penalties — but 
that is not the case for any other survey; and not all false answers are 
"willfully" false. 
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but the data for Incidental Indicators have already been gathered, and 
often even published, by other agencies and so can be utilized for EGI 
purposes at little extra expense to the educational system. In a 
larger perspective, though, the costs must still be borne, and It would 
not seem unreasonable for these other agencies to demand that the 
educational system share the costs If It Is to make use of the results, 
so this advantage may eventually be lost. Conversely, a survey Intended 
primarily for EGI purposes could be so designed as to meet the needs of 
other agencies at the same time, and they could therefore be asked to 
contribute to Its costs. Moreover, as has been pointed out, total costs 
could be held down by Item-sampling techniques and by rotating coverage 
of goal areas over a period of years. 

it it it it 

This series of comparisons leads to the fairly clear conclusion 
that, If educational progress Is to be measured by Indicators, 
Intentional Indicators are to be preferred. They have fewer disadvantages 
than Incidental Indicators, and those that they do have can be more easily 
overcome. This Is not to say that Indicators of any kind can do the job 
by themselves; It should be remembered that they are, at best, indirect 
measures. Direct measures. In the form of In-school tests, remain 
Indispensable for many purposes. Some combination of Indicators and tests 
Is probably the most advisable measurement strategy over the long run. 

Insofar as Indicators are to be used, the procedure suggested by the 
evidence of this Investigation Is to develop, for regular and routine adminis- 
tration, a set of survey Items designed specifically for measuring the level 
of attainment of New Jersey ^s educational goals. The survey should be 
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conducted either by an Independent agency of the state not directly 
Involved In education^ or else by a private contractor. Efforts 
should be made to persuade other states to administer the same Items » 
In order to provide comparative data* (The Education Commission 
of the States might be an appropriate channel for such efforts.) 

At the same tlme» It must be recognized that the measurement 
properties and possibilities of even Intentional Indicators leave much 
to be desired at the present time. Consequently, It would be wise to 
Initiate a program of research and development aimed at improving the 
reliability, the validity, and the comprehensiveness of the survey items, 
and at determining the kinds and sources of data that would help establish 
the range of reasonable levels of goal attainment. Where the state's 
sights should be set within the range is ultimately a matter for New 
Jersey's citizens to decide. 
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